To clarify:

Without this change cross-built gcc

... meaning build!=host.

Just want to make sure nobody reading along is thinking host!=target. What we're talking about here is ${stdenvBootstrapTools}/bin/gcc, not the compiler that builds it.

Okay, I think this is starting to make sense now. I need to run a bunch of builds overnight; assuming those all go well, and we can explain why this was a problem for powerpc64le but not for mips64el, I'll close #181802 in favor of this.

Unfortunately that means we will need to wait for Hydra to rebuild with it and then post a new bootstrap-files to tarballs.nixos.org ☹️

I've verified that this PR fixes the stack-protector issue in stdenv on powerpc64le, so I've closed #181802.

I've also rebuilt using a version of this PR which removes the &&hostPlatform==targetPlatform and passes --with-build-sysroot as suggested here, and have started a new bootstrap with that. No problems so far; if the build finishes I do think we should use that flag instead so we can simplify the conditional and so this bug won't resurface in canadian cross compiles.

I'm also re-running the bootstrap on mips64el just to make sure nothing broke. That may take a day or two unfortunately.

I've also rebuilt using a version of this PR which removes the &&hostPlatform==targetPlatform and passes --with-build-sysroot as suggested here, and have started a new bootstrap with that.

This bootstrap completed successfully.

Darwin did not like it:

impure path `//' used in link
collect2: error: ld returned 1 exit status

Will need to check why,

Don't immediately see how // sneaks into ld argument. Suspect linker script. Turned the change back to only cross-compilers and cross-built compilers.

Found -syslibroot // in darwin-specific gcc specs. We can safely skip it as not introducing impurity. Re-enabled --with-build-sysroot=/ on non-cross as well.

One more change:

Dropped mingw special case for no-libc build. Before the change we passed both --without-headers --with-native-system-headers-dir for no-libc gcc-static builds. This tricked darwin builds to find sys/sdt.h and fail inhibid_libc builds. Now all targets avoid passing native headers for gcc-static builds.

While at it fixed correct headers passing to --with-native-system-headers-dir= in host != target case: we were passing host's headers where intention was to pass target's headers. Noticed the mismatch as a build failure on pkgsCross.powernv.stdenv.cc on darwin where sys/sdt.h is present in host's headers (libSystem) but not target's headers (glibc).

Should be ready now.

I tested a few things:

• on x86_64-linux:
  • gcc (native)
  • pkgsMusl.hello, pkgsStatic.hello (less frequent toolchains)
  • pkgsCross.powernv.stdenv.cc, pkgsCross.mingw32.stdenv.cc (cross-compiler toolchains)
  • pkgsCross.powernv.gcc (cross-building compilers)
• on x86_64-darwin:
  • gcc (native)
  • pkgsMusl.hello, pkgsStatic.hello (less frequent toolchains)
  • pkgsCross.powernv.stdenv.cc, pkgsCross.mingw32.stdenv.cc, pkgsCross.aarch64-darwin.stdenv.cc (cross-compiler toolchains)
  • pkgsCross.powernv.gcc (cross-building compilers)

All cases seem to work and detect target libc features. darwin test was the best as it's host libc is very different from target linux libc. It made pre-existing headers mismatches very clear.

This is a very nicely done fix. I love the attention to not just making sure things work, but making sure things work for the same reasons across both cross and native. Thanks!!

Thank you! There is one minor fallout: not-quite-cross case of pkgsLLVM breakage. Proposed the fix as #182666.

This is 2 years too late, but --with-native-system-header-dir is burned into the compiler and is difficult to get rid of, so this PR breaks using an alternative --sysroot in the final compiler:

ignoring nonexistent directory "/nix/store/kzckb70l78jqfrahqh0abws9sq8i4l8p-linaro-libc/nix/store/p7hsyxycnq8azh9pr6p9q3wy3bwib6fr-glibc-2.40-36-dev/include"

I could work around the problem with -nostdinc and adding all the directories in the wrapper, but this is ugly. Semantically target_header_dir should be relative to the sysroot and I think we shouldn't change that.

Yup, --sysroot= handling is quite broken in nixpkgs. I don't think binutils-wrapper/cc-wrapper/patchelf understand any of that to work.

Also to make anything but --sysroot=/ work under nix something needs to place files in that sysroot. Right now nixpkgs dumps everything in a top-level /nix/store/$pkg hierarchy. Any attempt to use --sysroot= outside / breaks most of the paths. Building the compiler with a different --with-native-system-header-dir= for your case might be the least painful way in the short term.

Out of curiosity: how do you populate the sysroot? Do you use symlinks to /nix/store? Or something more heavyweight?

Out of curiosity: how do you populate the sysroot? Do you use symlinks to /nix/store? Or something more heavyweight?

My use case is to use a dump of the glibc headers and .sos from a vendor SDK (yup, very cursed indeed 😛). For a nixpkgs solution, I imagine symlinks would suffice for --with-build-sysroot=. On the cc-wrapper side, we can start off by generating a sysroot per platform, but this can be extended to allow composing multiple target platforms (imagine clang --target aarch64-unknown-linux-gnu just working).